Method of correcting projection image, projection system, and non-transitory computer-readable storage medium storing program

ABSTRACT

A method of correcting a projection image includes projecting, by a projector, a first image including a plurality of control points configured to correct the projection image on a projection surface, identifying at least one first control point from the plurality of control points based on a position of at least one of hands of a user in the first image when a first gesture operation by at least one of the hands of the user is detected, and projecting, by the projector, a second image obtained by moving a position of the at least one first control point from a first position to a second position based on a position of at least one of the hands of the user in the first image on the projection surface when a second gesture operation by at least one of the hands of the user is detected.

The present application is based on, and claims priority from JPApplication Serial Number 2022-092785, filed Jun. 8, 2022, thedisclosure of which is hereby incorporated by reference herein in itsentirety.

BACKGROUND 1. Technical Field

The present disclosure relates to a method of correcting a projectionimage, a projector, a projection system, and a non-transitorycomputer-readable storage medium storing a program.

2. Related Art

In the past, there has been known a technology of correcting aprojection image. For example, JP-A-2012-129594 (Document 1) discloses amethod of correcting positions of four corners of a projection image bymoving adjustment markers in accordance with a movement of a hand.

However, in the method disclosed by Document 1, a selection operation ofthe adjustment marker to be moved is not considered. Therefore, in themethod disclosed by Document 1, there can occur an erroneous operationthat the adjustment marker which the user does not intend to move movesto a position which the user does not intend in accordance with a motionof a hand of the user.

SUMMARY

A method of correcting a projection image according to the presentdisclosure includes projecting, by a projector, a first image includinga plurality of control points configured to correct the projection imageon a projection surface, identifying at least one first control pointfrom the plurality of control points based on a position of at least oneof hands of a user in the first image when a first gesture operation byat least one of the hands of the user is detected based on an outputvalue of a sensor, and projecting, by the projector, a second imageobtained by moving a position of the at least one first control pointfrom a first position to a second position based on a position of atleast one of the hands of the user in the first image on the projectionsurface when a second gesture operation by at least one of the hands ofthe user different from the first gesture operation is detected based onan output value of the sensor.

A projector according to the present disclosure includes at least oneprocessor configured to execute projecting a first image including aplurality of control points configured to correct a projection image ona projection surface, identifying at least one first control point fromthe plurality of control points based on a position of at least one ofhands of a user in the first image when a first gesture operation by atleast one of the hands of the user is detected based on an output valueof a sensor, and projecting a second image obtained by moving a positionof the at least one first control point from a first position to asecond position based on a position of at least one of the hands of theuser in the first image on the projection surface when a second gestureoperation by at least one of the hands of the user different from thefirst gesture operation is detected based on an output value of thesensor.

A projection system according to the present disclosure includes asensor, a control device configured to execute identifying at least onefirst control point from a plurality of control points configured tocorrect the projection image based on a position of at least one ofhands of a user in a first image which is projected on a projectionsurface, and which includes the plurality of control points, when afirst gesture operation by at least one of the hands of the user isdetected based on an output value of a sensor, and detecting a secondgesture operation by at least one of the hands of the user differentfrom the first gesture operation based on an output value of the sensor,and a projector configured to execute projecting the first image on theprojection surface, and projecting a second image obtained by moving aposition of the at least one first control point from a first positionto a second position based on a position of at least one of the hands ofthe user in the first image on the projection surface when the secondgesture operation is detected by the control device.

In a non-transitory computer-readable storage medium storing a programaccording to the present disclosure, the program is configured to makeat least one processor execute processing including projecting a firstimage including a plurality of control points configured to correct aprojection image from a projector on a projection surface, identifyingat least one first control point from the plurality of control pointsbased on a position of at least one of hands of a user in the firstimage when a first gesture operation by at least one of the hands of theuser is detected based on an output value of a sensor, and projecting asecond image obtained by moving a position of the at least one firstcontrol point from a first position to a second position based on aposition of at least one of the hands of the user in the first imagefrom the projector on the projection surface when a second gestureoperation by at least one of the hands of the user different from thefirst gesture operation is detected based on an output value of thesensor.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view showing an installation aspect of aprojector.

FIG. 2 is a side view showing the installation aspect of the projector.

FIG. 3 is a diagram showing a configuration of the projector.

FIG. 4 is a diagram showing an example of a point correction UI.

FIG. 5 is a diagram showing an example of the point correction UI.

FIG. 6 is a diagram showing an example of the point correction UI.

FIG. 7 is a diagram showing an example of the point correction UI.

FIG. 8 is a diagram showing an example of the point correction UI.

FIG. 9 is a flowchart showing an operation of the projector.

FIG. 10 is a diagram showing an installation aspect of the projector.

FIG. 11 is a diagram showing a configuration of a projection system.

FIG. 12 is a flowchart showing an operation of a control device.

DESCRIPTION OF EXEMPLARY EMBODIMENTS First Embodiment

First, a first embodiment will be described.

FIG. 1 is a perspective view showing an installation aspect of aprojector 1. FIG. 2 is a side view showing an installation aspect of theprojector 1.

The projector 1 generates image light corresponding to image data, andthen emits the image light thus generated on a projection surface 2.

The description will hereinafter be presented defining a normal line ofthe projection surface 2 as a Z axis, a vertical axis as a Y axis, andan axis perpendicular to the Z axis and the Y axis as an X axis. The Zaxis corresponds to a front-back direction in an installation state ofthe projector 1. A positive direction of the Z axis represents a forwarddirection. The Y axis corresponds to an up-down direction. A positivedirection of the Y axis represents an upward direction. The X axiscorresponds to a right-left direction. A positive direction of the Xaxis represents a rightward direction.

The projector 1 is installed in front of and above the projectionsurface 2, and emits the image light toward the projection surface 2. Aregion of the projection surface 2 to which the image light is emittedis hereinafter referred to as a projection region 3. On the projectionregion 3, there is projected the projection image by the image lightemitted by the projector 1.

It should be noted that the projection surface 2 can be a screen, or canalso be a wall surface of a building, or a plane of an installation.Further, the projection surface 2 is not limited to a plane, but canalso be a curved surface or a surface having asperity.

The projector 1 is provided with an interactive function. Theinteractive function means a function of detecting a position of theprojection surface 2 designated with a pointer, and then project animage corresponding to a position or a trajectory of the pointer on theprojection surface 2 based on the position thus detected, or adding achange to the image thus projected.

In the present embodiment, a finger 5 of a hand 4 is illustrated as thepointer.

The projector 1 emits detection light LT in a direction corresponding tothe projection surface 2. Therefore, when a tip of the finger 5 makescontact with the projection surface 2, the finger 5 blocks the detectionlight LT. The detection light LT collides with the finger 5 to bereflected, and a part of the reflected light of the detection light LTproceeds toward the projector 1 from the finger 5. The projector 1detects the reflected light which is the detection light LT reflected bythe finger 5 to thereby detect a position pointed by the finger 5. Theprojector 1 emits the detection light LT in the direction correspondingto the projection surface 2. Specifically, the direction correspondingto the projection surface 2 is a direction in which the reflected lightreflected by the finger 5 in a range HA can enter an infrared camera 17.The range HA means a range in the front-back direction, and a range inwhich a distance in the forward direction from the projection surface 2is equal to or shorter than a threshold value. The detection light LT isemitted in a range including at least a part of the projection surface2. In the present embodiment, the detection light LT is emitted in arange which covers at least an entire area of the projection region 3.It should be noted that the detection light LT is the light used fordetecting the pointer, and infrared light is used as the detection lightLT in the preset embodiment.

The infrared camera 17 is an example of a sensor in the presentdisclosure.

FIG. 3 is a diagram showing a configuration of the projector 1.

The projector 1 is provided with a first controller 10.

The first controller 10 is provided with a first processor 100 forexecuting a program such as a CPU (Central Processing Unit) or an MPU(Micro-Processing Unit) and a first memory 110, and controls each partof the projector 1. Functional units of the first processor 100 will bedescribed later.

The first controller 10 is an example of a controller in the presentdisclosure. The first processor 100 is an example of at least oneprocessor in the present disclosure.

The first memory 110 stores a program to be executed by the firstprocessor 100, and data to be processed by the first processor 100. Thefirst memory 110 stores a first control program 111 to be executed bythe first processor 100, first calibration data 112, second calibrationdata 113, and other various types of data. The first memory 110 has anonvolatile storage area. The first memory 110 can be provided with avolatile storage area to constitute a work area for the first processor100. The first memory 110 is formed of, for example, a ROM (Read OnlyMemory) and a RAM (Random Access Memory).

The first control program 111 is an example of a program in the presentdisclosure.

The first calibration data 112 is data of associating a taken image bythe infrared camera 17 and the projection region 3 with each other. Morespecifically, the first calibration data 112 is data for makingcoordinates of pixels in a coordinate system of the taken image by theinfrared camera 17 and coordinates of pixels in a coordinate system of aframe memory 12 correspond to each other.

The second calibration data 113 is data of associating the projectionregion 3 and each of control points 203 of a point correction UI 200described later with each other. More specifically, the secondcalibration data 113 is data for making the coordinates of the pixels inthe coordinate system of the frame memory 12 and coordinates of pixelscorresponding to the control points 203 of the point correction UI 200developed in the frame memory 12 correspond to each other.

The projector 1 is provided with a first interface 11.

The first interface 11 is provided with a connector, an interfacecircuit, and so on compliant with a predetermined communicationstandard. The first interface 11 transmits/receives the image data,control data, and so on to/from an external device in accordance withcontrol by the first controller 10, and in compliance with thepredetermined communication standard. It should be noted that the imagedata to be transmitted/received by the first interface 11 can be stillimage data, or moving image data, and can also be accompanied by audiodata.

The projector 1 is provided with the frame memory 12, and an imageprocessor 13.

The frame memory 12 and the image processor 13 are formed of, forexample, an integrated circuit. The integrated circuit includes an LSI,an ASIC (Application Specific Integrated Circuit), a PLD (ProgrammableLogic Device), an FPGA (Field-Programmable Gate Array), an SoC(System-on-a-Chip), and so on. Further, an analog circuit can beincluded as a part of a configuration of the integrated circuit, or itis also possible to adopt a configuration in which the first controller10 and the integrated circuit are combined with each other.

The frame memory 12 is provided with a plurality of banks. Each of thebanks has a storage capacity sufficient for writing one frame. The framememory 12 is formed of, for example, an SDRAM (Synchronous DynamicRandom Access Memory).

The image processor 13 performs image processing such as resolutionconversion processing, resizing processing, a correction of a distortionaberration, geometric correction processing, digital zoom processing,and image processing such as an adjustment of the tint and luminance ofthe image on the image data developed in the frame memory 12. The imageprocessor 13 executes the processing designated by the first controller10, and performs the processing using a parameter input from the firstcontroller 10 as needed. Further, it is possible for the image processor13 to execute two or more of the image processing described above incombination with each other.

Due to the control by the first controller 10, the image processor 13performs processing of superimposing a user interface related to theprojector 1 on an image represented by image data developed in the framememory 12. The image processor 13 superimposes the user interface forperforming a point correction on the image represented by the image datadeveloped in the frame memory 12. The point correction means acorrection of the projection image performed by a position adjustment ofthe control points 203 arranged in a grid-like pattern. The userinterface for performing the point correction is hereinafter describedas the point correction UI 200. In the point correction, a parameter ofa geometric correction is calculated based on a result of the pointcorrection, and the image processor 13 processes the image developed inthe frame memory 12 using that parameter.

When the superimposition of the point correction UI 200 is instructed bythe first controller 10, the image processor 13 superimposes the pointcorrection UI 200 on the image represented by the image data developedin the frame memory 12. Specifically, the image processor 13 combinesimage data of the point correction UI 200 with the image data developedin the frame memory 12. The image data thus combined is output to adriver 14. It should be noted that when the instruction of superimposingthe point correction UI 200 is not issued by the first controller 10,the image processor 13 outputs the image data developed in the framememory 12 to the driver 14 without combining the image data of the pointcorrection UI 200.

FIG. 4 is a diagram showing an example of the point correction UI 200.

The point correction UI 200 includes a plurality of vertical lines 201and a plurality of horizontal lines 202. The vertical lines 201 arearranged at regular intervals in the right-left direction. Thehorizontal lines 202 are arranged at regular intervals in the up-downdirection. Intersections between the vertical lines 203 and thehorizontal lines 202 are each the control point 203. The control points203 are points to be controlled by the user in order to perform thecorrection of the projection image. The position of each of the controlpoints 203 is adjusted by the user.

In the following description, when indicating the control point 203 atan N-th row and an M-th column, a description of 203-N-M is used. Thecharacters N and M each represent an integer equal to or greater than 1.The character N is defined as an integer representing a row numberincremented along a rightward direction. The character M is defined asan integer representing a column number incremented along a downwarddirection. For example, in the point correction UI 200 shown in FIG. 4 ,the control point 203 corresponding to an upper left corner is describedas the control point 203-1-1. Further, for example, in the pointcorrection UI 200 shown in FIG. 4 , the control point 203 correspondingto a lower right corner is described as the control point 203-11-8.

Although there is illustrated when the point correction UI 200 includesthe 11 vertical lines 201 and the 8 horizontal lines 202 in the presentembodiment, the number of the vertical lines 201 and the number of thehorizontal lines 202 included in the point correction UI 200 are onlyrequired to be plural, and are not limited to the numbers illustrated inthe present embodiment.

The projector 1 is provided with the driver 14 and a projection unit 15.

The driver 14 includes a light source drive circuit for putting a lightsource 151 on or off, and a light modulation device drive circuit. Thelight modulation device drive circuit drives a light modulation device152 in accordance with the control by the first controller 10 to draw animage frame by frame on light modulation elements provided to the lightmodulation device 152. To the light modulation device drive circuit,there is input image data corresponding to the respective primary colorsof R, G, and B from the image processor 13. The light modulation devicedrive circuit converts the image data input thereto into data signalssuitable for the operations of the liquid crystal panels as the lightmodulation elements provided to the light modulation device 152. Thelight modulation device drive circuit applies a voltage to each pixel ofeach of the liquid crystal panels based on the data signal thusconverted, and draws an image on each of the liquid crystal panels.

The projection unit 15 is provided with a light source 151, the lightmodulation device 152, and an optical unit 153.

The light source 151 is provided with a light source such as a xenonlamp, a super-high pressure mercury lamp, an LED, or a laser source.Further, the light source 151 can also be provided with a reflector andan auxiliary reflector for guiding light emitted by the light source tothe light modulation device 152.

The light modulation device 152 is provided with modulation elementssuch as transmissive liquid crystal panels. The light modulation device152 modulates the light entering the light modulation device 152 fromthe light source 151 to form the image light in accordance with the datasignals input from the drive circuit. The image light is typically colorimage light including visible light of three colors of red (R), green(G), and blue (B). The light modulation device 152 is not limited to thetransmissive liquid crystal panels, and can be, for example, reflectiveliquid crystal panels, or can also be DMDs (Digital MicromirrorDevices).

The optical unit 153 emits the image light formed by the lightmodulation device 152 to the projection surface 2 to form an image onthe projection surface 2. The optical unit 153 includes at least one ofa lens and a mirror. The optical unit 153 can also be provided with azoom mechanism for expanding or reducing the image projected on theprojection surface 2, and a focus adjustment mechanism for performing afocus adjustment.

An emitter 16 emits the detection light LT for detecting the finger 5 ina direction including the projection surface 2. In FIG. 3 , thedetection light LT is represented by a dotted line. The emitter 16 isprovided with an LD (Laser Diode) or an LED as the light source foremitting the infrared light. Further, it is also possible for theemitter 16 to be provided with an optical component for diffusing theinfrared light emitted by the light source toward the projection surface2.

The infrared camera 17 is provided with an imaging element such as aCMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge CoupledDevice) for receiving the detection light LT emitted by the finger 5.Further, the infrared camera 17 is provided with an optical system forforming an image on the imaging element, a diaphragm for limiting lightentering the imaging element, and so on.

The infrared camera 17 images a range including the projection surface 2to form a taken image. The infrared camera 17 receives the detectionlight LT emitted by the finger 5 to perform imaging. The image data ofthe taken image generated by the infrared camera 17 is output to thefirst controller 10.

The taken image is an example of an output value in the presentdisclosure.

The first processor 100 retrieves and then executes the first controlprogram 111 stored in the first memory 110 to thereby function as afirst operation detector 101 and a projection controller 102.

The first operation detector 101 detects a gesture operation by theuser. The first operation detector 101 detects a gesture operation basedon the taken image obtained by imaging by the infrared camera 17.

The first operation detector 101 detects a first gesture operation byone hand 4 of the user. In the present embodiment, the first gestureoperation is an operation of moving the hand 4 having a first shape intoa range HA. More particularly, the first gesture operation in thepresent embodiment is an operation of moving a part of the hand 4 havingthe first shape into the range HA. Further more particularly, the firstgesture operation in the present embodiment is an operation of movingthe finger 5 of the hand 4 having the first shape into the range HA. Itshould be noted that in the present embodiment, there is illustrated thehand 4 in the state of erecting one finger 5 as the hand 4 having thefirst shape. When a single image of the detection light LT shows up inthe taken image, the first operation detector 101 detects the firstgesture operation.

The first operation detector 101 detects a second gesture operation byone hand 4 of the user. In the present embodiment, the second gestureoperation is an operation of moving the hand 4 having a second shapedifferent from the first shape in the range HA as much as a distance noshorter than a first distance. More particularly, the second gestureoperation in the present embodiment is an operation of moving a part ofthe hand 4 having the second shape in the range HA as much as thedistance no shorter than the first distance. Further more particularly,the second gesture operation in the present embodiment is an operationof moving the finger 5 of the hand 4 having the second shape in therange HA as much as the distance no shorter than the first distance. Itshould be noted that in the present embodiment, there is illustrated thehand 4 in the state of erecting two fingers 5 as the hand 4 having thesecond shape. The first distance is a distance corresponding to apredetermined number of pixels in the taken image. When two images ofthe detection light LT show up in the taken image, a distance betweenthe two images is shorter than a predetermined number of pixels, and atthe same time, the two images move in the taken image as much as adistance no shorter than the pixels corresponding to the first distance,the first operation detector 101 detects the second gesture operation.

The first operation detector 101 detects a third gesture operation byone hand 4 of the user. In the present embodiment, the third gestureoperation is an operation of moving the hand 4 having the first shape inthe range HA as much as a distance no shorter than a second distance.More particularly, the third gesture operation in the present embodimentis an operation of moving a part of the hand 4 having the first shape inthe range HA as much as the distance no shorter than the seconddistance. Further more particularly, the third gesture operation in thepresent embodiment is an operation of moving the finger 5 of the hand 4having the first shape in the range HA as much as the distance noshorter than the second distance. The second distance is a distancecorresponding to a predetermined number of pixels in the taken image.The second distance can be the same distance as the first distance, orcan also be a different distance from the first distance. When a singleimage of the detection light LT shows up in the taken image, and at thesame time, the single image moves in the taken image as much as adistance no shorter than the pixels corresponding to the seconddistance, the first operation detector 101 detects the third gestureoperation.

The first operation detector 101 detects a fourth gesture operation byone hand 4 of the user. In the present embodiment, the fourth gestureoperation is an operation in which an operation of moving the hand 4having the first shape into the range HA and then moving the hand 4outside the range HA is performed L times in a predetermined period.More particularly, the fourth gesture operation in the presentembodiment is an operation in which an operation of moving a part of thehand 4 having the first shape into the range HA and then moving the partof the hand 4 outside the range HA is performed L times in thepredetermined period. Further more particularly, the fourth gestureoperation in the present embodiment is an operation in which anoperation of moving the finger 5 of the hand 4 having the first shapeinto the range HA and then moving the finger 5 of the hand 4 outside therange HA is performed L times in the predetermined period. Here, thecharacter L is an integer equal to or greater than 1. In the presentembodiment, there is illustrated when the character L is 2. When thesingle image of the detection light LT shows up twice in a predeterminedrange on the taken image in a predetermined period, the first operationdetector 101 detects the fourth gesture operation.

The projection controller 102 controls the image processor 13, thedriver 14, and so on to project the projection image on the projectionsurface 2. Specifically, the projection controller 102 makes the imageprocessor 13 process the image data developed in the frame memory 12. Onthis occasion, the projection controller 102 retrieves a parameter whichis necessary for the image processor 13 to perform the processing fromthe first memory 110, and then outputs the parameter to the imageprocessor 13. Further, the projection controller 102 controls the driver14 to make the light source drive circuit put the light source 151 on,make the light modulation device drive circuit to drive the lightmodulation device 152, and make the projection unit 15 emit the imagelight to display an image on the projection surface 2. Further, theprojection controller 102 controls the optical unit 153 to start up themotor to control the zoom and the focus of the optical unit 153.

The projection controller 102 in the present embodiment changes adisplay mode of the control points 203 based on the gesture operationdetected by the first operation detector 101 during a period in whichthe projector 1 projects the point correction UI 200. The projectioncontroller 102 instructs the image processor 13 to change the displaymode of the control points 203 to thereby change the display mode of thecontrol points 203 based on the gesture operation detected by the firstoperation detector 101.

The projection controller 102 in the present embodiment moves thepositions of the control points 203 based on the gesture operationdetected by the first operation detector 101 during the period in whichthe projector 1 projects the point correction UI 200. The projectioncontroller 102 instructs displacement of the control points 203 to theimage processor 13 to thereby move the positions of the control points203. It should be noted that the instruction output from the projectioncontroller 102 to the image processor 13 includes information of amoving direction and a moving distance.

Further, when moving the positions of the control points 203 in thepoint correction UI 200, the projection controller 102 of the presentembodiment corrects the projection image in accordance with thedisplacement of the control points 203. The projection controller 102performs an instruction of moving a pixel corresponding to the positionof the control point 203 as a movement target, and peripheral pixelslocated on the periphery of that pixel on the image processor 13 tothereby correct the projection image.

First, the point correction UI 200 when the first operation detector 101has detected the first gesture operation will be described.

FIG. 5 is a diagram showing an example of the point correction UI 200when the first operation detector 101 has detected the first gestureoperation.

When the first operation detector 101 has detected the first gestureoperation, the projection controller 102 identifies the control point203 selected by the user based on the position of the hand 4 in thetaken image. Hereinafter, the control point 203 selected by the user isreferred to as a first control point, and the control point 203 which isnot selected by the user is referred to as a second control point. Theprojection controller 102 identifies the first control point based onthe taken image by the infrared camera 17. The projection controller 102identifies a coordinate in the frame memory 12 corresponding to aposition of the image of the detection light LT in the taken image bythe infrared camera 17 with reference to the first calibration data 112.Then, the projection controller 102 refers to the second calibrationdata 113, and when the coordinate in the frame memory 12 thus identifiedis the coordinate corresponding to the control point 203, the projectioncontroller 102 identifies the control point 203 corresponding to thecoordinate in the frame memory thus identified as the first controlpoint.

The projection controller 102 changes the display mode of the controlpoint 203 identified as the first control point from a second displaymode to a first display mode different from the second display mode. Thesecond display mode is a display mode representing a selection cancelstate as a state in which the selection is not made. The first displaymode is a display mode representing a selection state as a state inwhich the selection is made. The first display mode and the seconddisplay mode are different from each other in color, shape, size, or thelike. It should be noted that the display mode of the control point 203as the second control point is the second display mode.

In FIG. 5 , there is shown when the user selects the control point203-4-1 using the first gesture operation when all of the control points203 are the second control points. In the case of FIG. 5 , theprojection controller 102 identifies the control point 203-4-1 as thefirst control point, and then changes the display mode of the controlpoint 203-4-1 from the second display mode to the first display mode.

Then, the point correction UI 200 when the first operation detector 101has detected the third gesture operation will be described.

FIG. 6 is a diagram showing an example of the point correction UI 200when the first operation detector 101 has detected the third gestureoperation.

When the first operation detector 101 has detected the third gestureoperation, the projection controller 102 identifies a setting area SA inthe point correction UI 200 with reference to the first calibration data112. The setting area SA is an area on the point correction UI 200 setby the user with the third gesture operation. For example, theprojection controller 102 identifies a rectangular area, which takes amoving start position and a moving end position of the hand 4 in thethird gesture operation as corners opposed to each other, as the settingarea SA. Then, the projection controller 102 identifies the controlpoint 203 included in the setting area SA thus identified as the firstcontrol point with reference to the second calibration data 113. Whenthe projection controller 102 identifies the first control point, theprojection controller 102 changes the display mode of the control point203 identified as the first control point from the second display modeto the first display mode.

In FIG. 6 , there is shown when the user sets the setting area SAincluding the 10 control points 203 on the point correction UI 200 whenall of the control points 203 are the second control points. In FIG. 6 ,the control points 203-4-2, 203-5-2, 203-6-2, 203-7-2, 203-8-2, 203-4-3,203-5-3, 203-6-3, 203-7-3, and 203-8-3 are included in the setting areaSA. In the case of FIG. 6 , the projection controller 102 identifiesthese 10 control points 203 as the first control points, and thenchanges the display mode of these 10 control points 203 identified asthe first control points from the second display mode to the firstdisplay mode.

Then, the point correction UI 200 when the first operation detector 101has detected the second gesture operation will be described.

FIG. 7 is a diagram showing an example of the point correction UI 200when the first operation detector 101 has detected the second gestureoperation.

When the first operation detector 101 has detected the second gestureoperation, the projection controller 102 moves the positions of thefirst control points from first positions to second positions inaccordance with the movement of the hand 4 in the second gestureoperation. The first positions are positions before the movement by thesecond gesture operation, and the second positions are positions towhich the first control points are moved by the second gestureoperation. The projection controller 102 identifies the moving distanceand the moving direction of the hand 4 in the second gesture operationwith reference to the first calibration data 112, and moves thepositions of the first control points in accordance with the movingdistance and the moving direction thus identified. The projectioncontroller 102 corrects the projection image in accordance with thedisplacement of the first control points. It should be noted that thepoint correction UI 200 in which the positions of the first controlpoints are the first positions is an example of a first image in thepresent disclosure, and the point correction UI 200 in which thepositions of the first control points are the second positions is anexample of a second image in the present disclosure.

In FIG. 7 , there is shown when the second gesture operation of movingtoward a lower right direction is detected when the control points203-4-2, 203-5-2, 203-6-2, 203-7-2, 203-8-2, 203-4-3, 203-5-3, 203-6-3,203-7-3, and 203-8-3 are the first control points. In the case of FIG. 7, the projection controller 102 moves the positions of these 10 controlpoints 203 toward the lower right direction in the point correction UI200.

Then, the point correction UI 200 when the first operation detector 101has detected the fourth gesture operation will be described.

FIG. 8 is a diagram showing an example of the point correction UI 200when the first operation detector 101 has detected the fourth gestureoperation.

The projection controller 102 changes the first control points to thesecond control points when the first operation detector 101 has detectedthe fourth gesture operation when the point correction UI 200 includesthe first control points.

In FIG. 8 , there is shown when the first operation detector 101 hasdetected the fourth gesture operation when the control points 203-4-2,203-5-2, 203-6-2, 203-7-2, 203-8-2, 203-4-3, 203-5-3, 203-6-3, 203-7-3,and 203-8-3 are the first control points. In the case of FIG. 8 , theprojection controller 102 changes these 10 first control points to thesecond control points.

Then, an operation of the projector 1 when projecting the pointcorrection UI 200 will be described.

FIG. 9 is a flowchart showing the operation of the projector 1.

The projection controller 102 determines (step SA1) whether or not thefirst operation detector 101 has detected the first gesture operation.

When the projection controller 102 determines that the first operationdetector 101 has detected the first gesture operation (YES in the stepSA1), the projection controller 102 identifies (step SA2) the controlpoint 203 selected with the first gesture operation as the first controlpoint, and then the projection controller 102 changes (step SA3) thedisplay mode of the control point 203 thus identified as the firstcontrol point to the first display mode, and then performs theprocessing in the step SA7.

Going back to the description of the step SA1, when the projectioncontroller 102 determines that the first operation detector 101 has notdetected the first gesture operation (NO in the step SA1), theprojection controller 102 determines (step SA4) whether or not the firstoperation detector 101 has detected the third gesture operation.

When the projection controller 102 determines that the first operationdetector 101 has not detected the third gesture operation (NO in thestep SA4), the projection controller 102 performs the processing in thestep SA7.

In contrast, when the projection controller 102 determines that thefirst operation detector 101 has detected the third gesture operation(YES in the step SA4), the projection controller 102 identifies (stepSA5) the control point 203 included in the setting area SA as the firstcontrol point. Subsequently, the projection controller 102 changes (stepSA6) the display mode of the control point 203 identified as the firstcontrol point to the first display mode.

Subsequently, the projection controller 102 determines (step SA7)whether or not the first operation detector 101 has detected the secondgesture operation.

When the projection controller 102 determines that the first operationdetector 101 has detected the second gesture operation (YES in the stepSA7), the projection controller 102 moves the position of the firstcontrol point, and further, corrects the projection image in accordancewith the displacement of the first control point (step SA8).

In contrast, when the projection controller 102 determines that thefirst operation detector 101 has not detected the second gestureoperation (NO in the step SA7), the projection controller 102 determines(step SA9) whether or not the first operation detector 101 has detectedthe fourth gesture operation.

When the projection controller 102 determines that the first operationdetector 101 has not detected the fourth gesture operation (NO in thestep SA9), the projection controller 102 performs the determination inthe step SA1 once again.

In contrast, when the projection controller 102 determines that thefirst operation detector 101 has detected the fourth gesture operation(YES in the step SA9), the projection controller 102 changes (step SA10)the first control point to the second control point.

When the projection controller 102 performs the processing in the stepSA8 or the step SA10, the projection controller 102 returns the processto the step SA1 to perform the processing in the step SA1 and thesubsequent steps once again.

Second Embodiment

Then, a second embodiment will be described.

FIG. 10 is a diagram showing an example of an installation aspect of aprojector 6 in the second embodiment. FIG. 11 is a diagram showing aconfiguration of a projection system 1000 equipped with the projector 6in the second embodiment.

As shown in FIG. 10 , the projector 6 according to the second embodimentemits the image light to the projection surface 2 located behind theprojector 6. The projector 6 according to the second embodiment iscoupled to a control device 7, and performs projection of the pointcorrection UI 200, a change of the display mode of the control point203, a displacement of the control point 203, and a correction of theprojection image corresponding to the displacement of the control point203 in accordance with control by the control device 7.

The projection system 1000 equipped with the projector 6 according tothe second embodiment is provided with the projector 6, the controldevice 7, and a color camera 8.

The color camera 8 is an example of the sensor in the presentdisclosure.

As shown in FIG. 10 , the color camera 8 is installed so as to take animage in front of the projector 6. The color camera 8 is coupled to thecontrol device 7 to transmit image data of the taken image obtained byimaging to the control device 7.

The control device 7 is provided with a second processor 700 forexecuting a program such as a CPU or an MPU, and a second memory 710,and controls each part of the control device 7. Functional units of thesecond processor 700 will be described later. The second processor 700is an example of at least one processor in the present disclosure.

The second memory 710 stores a program to be executed by the secondprocessor 700, and data to be processed by the second processor 700. Thesecond memory 710 stores a second control program 711 to be executed bythe second processor 700, the second calibration data 113, thirdcalibration data 712, and other various types of data. The second memory710 has a nonvolatile storage area. The second memory 710 can beprovided with a volatile storage area to form a work area for the secondprocessor 700. The second memory 710 is formed of, for example, a ROM(Read Only Memory) and a RAM (Random Access Memory).

The third calibration data 712 is data of associating a taken image bythe color camera 8 and the projection region 3 with each other. Morespecifically, the third calibration data 712 is data for makingcoordinates of pixels in a coordinate system of the taken image by thecolor camera 8 and coordinates of pixels in the coordinate system of theframe memory 12 of the projector 6 correspond to each other.

The control device 7 is provided with a second interface 71 and a thirdinterface 72.

The second interface 71 is an interface provided with hardware compliantwith a predetermined communication standard such as a communicationcircuit, and communicates with the color camera 8.

The third interface 72 is an interface provided with hardware compliantwith a predetermined communication standard such as a communicationcircuit, and communicates with the projector 6.

The second processor 700 functions as a second operation detector 701and a projector controller 702.

The second operation detector 701 detects the gesture operation by theuser. The second operation detector 701 detects the gesture operationbased on the taken image by the color camera 8.

The second operation detector 701 detects the first gesture operation.The first gesture operation in the present embodiment is the same as thefirst gesture operation described in the first embodiment. When an imageof the hand 4 having the first shape no smaller than a predeterminedarea shows up in the taken image, the second operation detector 701detects the first gesture operation. The predetermined area is the areawhich defines whether or not the position of the hand 4 is within therange HA, and is determined in advance by a test or a simulation. Itshould be noted that the second operation detector 701 detects the shapeof the hand 4 showing up in the taken image by the color camera 8 usinga predetermined method such as pattern matching or a machine learningresult.

The second operation detector 701 detects the second gesture operation.The second gesture operation in the present embodiment is the same asthe second gesture operation described in the first embodiment. When animage of the hand 4 having the second shape no smaller than apredetermined area moves in the taken image as much as a distance noshorter than the pixels corresponding to the first distance, the secondoperation detector 701 detects the second gesture operation.

The second operation detector 701 detects the third gesture operation.The third gesture operation in the present embodiment is the same as thethird gesture operation described in the first embodiment. When theimage of the hand 4 having the first shape no smaller than thepredetermined area moves in the taken image as much as a distance noshorter than the pixels corresponding to the second distance, the secondoperation detector 701 detects the third gesture operation.

The second operation detector 701 detects the fourth gesture operation.The fourth gesture operation in the present embodiment is the same asthe fourth gesture operation described in the first embodiment. When theimage of the hand 4 having the first shape no smaller than thepredetermined area shows up within a predetermined range on the takenimage twice in a predetermined period, the second operation detector 701detects the fourth gesture operation.

The projector controller 702 communicates with the projector 6 via thethird interface 72. The projector 702 transmits the control data to theprojector 6 to thereby control the projector 6. The projector controller702 makes the projector 6 project the point correction UI 200.

When the second operation detector 701 has detected the first gestureoperation, the projector controller 702 identifies the control point 203selected by the user based on the position of the hand 4 in the takenimage by the color camera 8. The projector controller 702 identifies acoordinate in the frame memory 12 corresponding to the position of theimage of the hand 4 in the taken image by the color camera 8 withreference to the third calibration data 712. Then, the projectorcontroller 702 refers to the second calibration data 113, and when thecoordinate in the frame memory 12 thus identified is the coordinatecorresponding to the control point 203, the projection controller 102identifies the control point 203 corresponding to the coordinate in theframe memory 12 thus identified as the first control point.

When the projector controller 702 identifies the first control point,the projector controller 702 transmits control data for changing thedisplay mode of the control point 203 identified as the first controlpoint from the second display mode to the first display mode to theprojector 6. The control data includes the coordinate in the framememory 12 corresponding to the control point 203 as a change target.

When the second operation detector 701 has detected the third gestureoperation, the projector controller 702 identifies the setting area SAin the point correction UI 200 with reference to the third calibrationdata 712. Then, the projector controller 702 identifies the controlpoint 203 included in the setting area SA thus identified as the firstcontrol point with reference to the second calibration data 113.

When the projector controller 702 identifies the first control point,the projector controller 702 transmits the control data for changing thedisplay mode of the control point 203 identified as the first controlpoint from the second display mode to the first display mode to theprojector 6.

When the second operation detector 701 has detected the second gestureoperation, the projector controller 702 moves the position of the firstcontrol point from the first position to the second position inaccordance with the movement of the hand 4 in the second gestureoperation. The projector controller 702 identifies the moving distanceand the moving direction of the hand 4 in the second gesture operationwith reference to the first calibration data 112, and moves the positionof the first control point in accordance with the moving distance andthe moving direction thus identified. The projector controller 702transmits the control data to the projector 6 to thereby move theposition of the first control point. The control data is data for movingthe position of the first control point, and at the same time,correcting the projection image in accordance with the displacement ofthe first control point. In the control data, there are described thecoordinate in the frame memory 12 corresponding to the control point 203as the movement target, the moving direction, and the moving distance asthe information.

When the second operation detector 701 has detected the fourth gestureoperation, the projector controller 702 makes the projector 6 change thefirst control point to the second control point. The projectorcontroller 702 transmits the control data for performing the change tothe second control point to the projector 6 to thereby make theprojector 6 change the first control point to the second control point.

As shown in FIG. 11 , the projector 6 is provided with substantially thesame configuration as that of the projector 1 according to the firstembodiment. It should be noted that regarding the configuration of theprojector 6 according to the second embodiment, the same constituents asin the first embodiment are denoted by the same reference symbols, andthe detailed description thereof will be omitted.

As shown in FIG. 11 , the first processor 100 of the projector 6functions as the projection controller 102.

The projection controller 102 in the second embodiment projects thepoint correction UI 200 in accordance with the control data transmittedby the control device 7. Further, the projection controller 102 in thesecond embodiment changes the display mode of the control point 203identified as the first control point to the first display mode inaccordance with the control data transmitted by the control device 7.Further, the projection controller 102 in the second embodiment movesthe position of the first control point, and at the same time, correctsthe projection image so as to correspond to the displacement of thefirst control point in accordance with the control data transmitted bythe control device 7. Further, the projection controller 102 in thesecond embodiment changes the first control point to the second controlpoint in accordance with the control data transmitted by the controldevice 7.

Then, an operation of the control device 7 when the projector 6 projectsthe point correction UI 200 will be described.

FIG. 12 is a flowchart showing the operation of the control device 7.

The projector controller 702 determines (step SB1) whether or not thesecond operation detector 701 has detected the first gesture operation.

When the projector controller 702 determines that the second operationdetector 701 has detected the first gesture operation (YES in the stepSB1), the projector controller 702 identifies (step SB2) the controlpoint 203 selected by the first gesture operation as the first controlpoint. Then, the projector controller 702 transmits (step SB3) thecontrol data of changing the display mode of the control point 203identified as the first control point to the first display mode to theprojector 6 via the third interface 72, and then performs the processingin the step SB7.

Going back to the description of the step SB1, when the projectorcontroller 702 determines that the second operation detector 701 has notdetected the first gesture operation (NO in the step SB1), the projectorcontroller 702 determines (step SB4) whether or not the second operationdetector 701 has detected the third gesture operation.

When the projector controller 702 determines that the second operationdetector 701 has not detected the third gesture operation (NO in thestep SB4), the projector controller 702 performs the processing in thestep SB7.

In contrast, when the projector controller 702 determines that thesecond operation detector 701 has detected the third gesture operation(YES in the step SB4), the projector controller 702 identifies (stepSB5) the control point 203 included in the setting area SA as the firstcontrol point. Subsequently, the projector controller 702 transmits(step SB6) the control data of changing the display mode of the controlpoint 203 identified as the first control point to the first displaymode to the projector 6.

Then, the projector controller 702 determines (step SB7) whether or notthe second operation detector 701 has detected the second gestureoperation.

When the projector controller 702 determines that the second operationdetector 701 has detected the second gesture operation (YES in the stepSB7), the projector controller 702 transmits (step SB8) the control dataof moving the position of the first control point, and further,correcting the projection image in accordance with the displacement ofthe first control point to the projector 6.

In contrast, when the projector controller 702 determines that thesecond operation detector 701 has not detected the second gestureoperation (NO in the step SB7), the projector controller 702 determines(step SB9) whether or not the second operation detector 701 has detectedthe fourth gesture operation.

When the projector controller 702 determines that the second operationdetector 701 has not detected the fourth gesture operation (NO in thestep SB9), the projector controller 702 returns the process to the stepSB1.

In contrast, when the projector controller 702 determines that thesecond operation detector 701 has detected the fourth gesture operation(YES in the step SB9), the projector controller 702 transmits (stepSB10) the control data of changing the first control point to the secondcontrol point to the projector 6.

When the projector controller 702 performs the processing in the stepSB8 or the step SB10, the projector controller 702 returns the processto the step SB1 to perform the processing in the step SB1 and thesubsequent steps once again.

Other Embodiments

The embodiments described above are each a preferred embodiment of thepresent disclosure. It should be noted that the present disclosure isnot limited to the embodiments described above, but can be implementedwith a variety of modifications within the scope or the spirit of thepresent disclosure.

In the first embodiment described above, there is illustrated theinfrared camera 17 as the sensor in the present disclosure, and in thesecond embodiment described above, there is illustrated the color camera8 as the sensor in the present disclosure. However, the sensor in thepresent disclosure is not limited to the cameras illustrated in theembodiments. As the sensor in the present disclosure, any sensors can beadopted as long as the first processor 100 or the second processor 700can identify the gesture operation of the user with those sensors, andit is possible to adopt, for example, a temperature sensor or a rangesensor.

In each of the embodiments described above, the state of erecting one ofthe fingers 5 is illustrated as the first shape, and the state oferecting two of the fingers 5 is illustrated as the second shape.However, it is sufficient for the first shape and the second shape to beshapes different from each other, and the number of fingers 5 to beerected is not limited to the embodiments described above. In otherwords, it is sufficient for the first shape to be a state of erecting Xfingers, and it is sufficient for the second shape to be a state oferecting Y fingers, wherein Y is different from X. Here, X and Y areeach an integer no smaller than 0 and no greater than 5.

In each of the embodiments described above, the operation of moving asingle hand 4 having the first shape into the range HA is illustrated asthe first gesture operation, and the operation of moving a single hand 4having the second shape within the range HA as much as the distance noshorter than the first distance is illustrated as the second gestureoperation. However, it is sufficient for the first gesture operation andthe second gesture operation to be different in shape of the hand 4 fromeach other, and the number of the hands 4, the number of the fingers 5,the shape of the finger 5, and so on are not limited to those in theembodiments described above.

For example, in other embodiments, it is possible to define the firstgesture operation as an operation of moving the right hand into therange HA, and define the second gesture operation as an operation ofmoving the left hand in the range HA as much as a distance no shorterthan the first distance.

Further, for example, in other embodiments, it is possible to define thefirst gesture operation as an operation of moving the hand 4 in thestate of not erecting the finger 5 into the range HA, and define thesecond gesture operation as an operation of moving the hand 4 in thestate of erecting an index finger and a middle finger alone in the rangeHA as much as a distance no shorter than the first distance.

Further, for example, in other embodiments, it is possible to define thefirst gesture operation as an operation of moving the hand 4 with one ofthe fingers 5 stretched straight into the range HA, and define thesecond gesture operation as an operation of moving the hand 4 with oneof the fingers 5 bent in the range HA as much as a distance no shorterthan the first distance.

Further, for example, in other embodiments, it is possible to define thefirst gesture operation as an operation of moving one hand into therange HA, and define the second gesture operation as an operation ofmoving the both hands in the range HA as much as a distance no shorterthan the first distance.

In the second embodiment described above, there is described when thecolor camera 8 is disposed outside the projector 6, but it is possibleto adopt a configuration in which the color camera 8 is installed in theprojector 6.

Further, it is possible to realize the functions of the first processor100 and the second processor 700 with a plurality of processors, or asemiconductor chip.

Further, the functional units of the projector 1 shown in FIG. 3 and thefunctional units of the projector 6 shown in FIG. 11 are thoserepresenting the functional configuration thereof, and the specificinstallation aspects are not particularly limited. In other words, it isnot necessarily required to install the hardware individuallycorresponding to each of the functional units, and it is obviouslypossible to adopt a configuration of realizing the functions of theplurality of functional units by a single processor executing a program.Further, a part of the function realized by software in the embodimentsdescribed above can also be realized by hardware, and a part of thefunction realized by hardware can also be realized by software. Besidesthe above, the specific detailed configuration of each of other units ofthe projectors 1, 6 can arbitrarily be modified within the scope or thespirit of the present disclosure.

Further, the functional units of the control device 7 shown in FIG. 11are those representing the functional configurations, and the specificinstallation aspects are not particularly limited. In other words, it isnot necessarily required to install the hardware individuallycorresponding to each of the functional units, and it is obviouslypossible to adopt a configuration of realizing the functions of theplurality of functional units by a single processor executing a program.Further, a part of the function realized by software in the embodimentsdescribed above can also be realized by hardware, and a part of thefunction realized by hardware can also be realized by software. Besidesthe above, the specific detailed configuration of each of other units ofthe control device 7 can arbitrarily be modified within the scope or thespirit of the present disclosure.

Further, the processing units of the flowchart shown in FIG. 9 areobtained by dividing the processing of the projector 1 in accordancewith major processing contents in order to make the processing of theprojector 1 easy to understand, and the present disclosure is not at alllimited by the way of dividing the processing into the processing unitsor the names of the processing units. It is possible to divide theprocessing into a larger number of unit steps in accordance with theprocessing contents. Further, it is also possible to divide theprocessing so that each of the unit steps includes a larger amount ofprocessing. Further, the order of the steps can arbitrarily be exchangedwithin a range in which no problem is posed in the scope or the spiritof the present disclosure.

Further, the processing units of the flowchart shown in FIG. 12 areobtained by dividing the processing of the control device 7 inaccordance with major processing contents in order to make theprocessing of the control device 7 easy to understand, and the presentdisclosure is not at all limited by the way of dividing the processinginto the processing units or the names of the processing units. It ispossible to divide the processing into a larger number of unit steps inaccordance with the processing contents. Further, it is also possible todivide the processing so that each of the unit steps includes a largeramount of processing. Further, the order of the steps can arbitrarily beexchanged within a range in which no problem is posed in the scope orthe spirit of the present disclosure.

Further, when realizing the correction method of the projection imageusing the first processor 100 provided to the projector 1, it is alsopossible to configure the program to be executed by the first processor100 as an aspect of a recording medium, or an aspect of a transmissionmedium for transmitting the program. As the recording medium, there canbe used a magnetic or optical recording medium, or a semiconductormemory device. Specifically, there can be cited a portable or rigidrecording medium such as a flexible disk, an HDD (Hard Disk Drive), aCD-ROM, a DVD, a Blu-ray disc, a magnetooptic disc, a flash memory, or acard-type recording medium. Further, the recording medium describedabove can also be a RAM, or a nonvolatile storage device such as a ROMor the HDD as an internal storage device provided to a server device.Blu-ray is a registered trademark.

Further, when realizing the correction method of the projection imageusing the projection system 1000, it is also possible to configure theprogram to be executed by the first processor 100 and the secondprocessor 700 as an aspect of a recording medium, or an aspect of atransmission medium for transmitting the program. As the recordingmedium, there can be used a magnetic or optical recording medium, or asemiconductor memory device.

Conclusion of Present Disclosure

Hereinafter, the conclusion of the present disclosure willsupplementarily be noted.

Supplementary Note 1

A method of correcting a projection image including projecting, by aprojector, a first image including a plurality of control pointsconfigured to correct the projection image on a projection surface,identifying at least one first control point from the plurality ofcontrol points based on a position of at least one of hands of a user inthe first image when a first gesture operation by at least one of thehands of the user is detected based on an output value of a sensor, andprojecting, by the projector, a second image obtained by moving aposition of the at least one first control point from a first positionto a second position based on a position of at least one of the hands ofthe user in the first image on the projection surface when a secondgesture operation by at least one of the hands of the user differentfrom the first gesture operation is detected based on an output value ofthe sensor.

According to the above, the operation of selecting the control point tobe the movement target and the operation of moving the control point areassociated with the respective gesture operations different from eachother. Therefore, it is possible to prevent an occurrence of anerroneous operation that the control point unintended by the user movesto a position unintended by the user in the correction of the projectionimage.

Supplementary Note 2

The method of correcting the projection image described in SupplementaryNote 1, wherein the first gesture operation is an operation of moving atleast one of the hands of the user having a first shape into a range inwhich a distance from the projection surface is equal to or shorter thana threshold value, and the second gesture operation is an operation ofmoving at least one of the hands of the user having a second shapedifferent from the first shape in the range as much as a distance equalto or longer than a first distance.

According to the above, since the shape of the hand in the first gestureoperation and the shape of the hand in the second gesture operation aredifferent from each other, and at the same time, the first gestureoperation and the second gesture operation are different from eachother, it is possible to further prevent the occurrence of the erroneousoperation that the control point unintended by the user moves to aposition unintended by the user in the correction of the projectionimage.

Supplementary Note 3

The method of correcting the projection image described in SupplementaryNote 2, wherein the identifying at least one first control pointincludes when a third gesture operation is detected based on an outputvalue of the sensor, identifying a setting area set by the third gestureoperation in the first image, and identifying a control point includedin the setting area out of the plurality of control points as the atleast one first control point, and the third gesture operation is anoperation of moving at least one of the hands of the user having thefirst shape in the range as much as a distance equal to or longer than asecond distance.

According to the above, it becomes possible to select a plurality ofcontrol points included in the setting area in a lump with the thirdgesture operation. Therefore, in the correction of the projection image,it is not necessary for the user to select two or more control pointsone by one. Therefore, it is possible to prevent an occurrence of anerroneous operation that the control point unintended by the user movesto a position unintended by the user in the correction of the projectionimage, and at the same time, it is possible to enhance the convenienceof the user.

Supplementary Note 4

The method of correcting the projection image described in one ofSupplementary Note 2 and Supplementary Note 3 further including changingthe at least one first control point to a second control point when afourth gesture operation is detected based on an output value of thesensor, wherein the fourth gesture operation is an operation of movingat least one of the hands of the user having the first shape into therange, and then moving the at least one of the hands of the user havingthe first shape outside the range within a predetermined period.

According to the above, it is possible to set the control point as notto be the movement target using a gesture operation different from thegesture operation of selecting the control point and the gestureoperation of moving the control point. Therefore, it is possible tofurther prevent an occurrence of the erroneous operation that thecontrol point unintended by the user moves to a position unintended bythe user in the correction of the projection image.

Supplementary Note 5

The method of correcting the projection image described in SupplementaryNote 4, wherein the first control point is a control point in a firstdisplay mode, and the second control point is a control point in asecond display mode different from the first display mode.

According to the above, it is possible for the user to easily figure outwhich control point is the control point to be moved by the gestureoperation. Therefore, it is possible to prevent an occurrence of theerroneous operation that the control point unintended by the user movesto a position unintended by the user in the correction of the projectionimage, and at the same time, it is possible to enhance the convenienceof the user.

Supplementary Note 6

The method of correcting the projection image described in any oneSupplementary Note 1 through Supplementary Note 5, wherein the firstshape is a state of erecting X fingers, the second shape is a state oferecting Y fingers, the X is an integer that is equal to or more than 0and that is equal to or less than 5, the Y is an integer that is equalto or more than 0 and that is equal to or less than 5, and the X isdifferent from the Y.

According to the above, by making the number of fingers of the handdifferent, it is possible for the user to appropriately distinguish thegesture operation of selecting the control point and the gestureoperation of moving the control point from each other. Therefore, it ispossible to prevent an occurrence of the erroneous operation that thecontrol point unintended by the user moves to a position unintended bythe user in the correction of the projection image, and at the sametime, it is possible to enhance the convenience of the user.

Supplementary Note 7

A projector including at least one processor configured to executeprojecting a first image including a plurality of control pointsconfigured to correct a projection image on a projection surface,identifying at least one first control point from the plurality ofcontrol points based on a position of at least one of hands of a user inthe first image when a first gesture operation by at least one of thehands of the user is detected based on an output value of a sensor, andprojecting a second image obtained by moving a position of the at leastone first control point from a first position to a second position basedon a position of at least one of the hands of the user in the firstimage on the projection surface when a second gesture operation by atleast one of the hands of the user different from the first gestureoperation is detected based on an output value of the sensor.

According to the above, there are exerted substantially the sameadvantages as those of the method of correcting the projection imageaccording to Supplementary Note 1.

Supplementary Note 8

A projection system including a sensor, a control device configured toexecute identifying at least one first control point from a plurality ofcontrol points configured to correct the projection image based on aposition of at least one of hands of a user in a first image which isprojected on a projection surface, and which includes the plurality ofcontrol points, when a first gesture operation by at least one of thehands of the user is detected based on an output value of a sensor, anddetecting a second gesture operation by at least one of the hands of theuser different from the first gesture operation based on an output valueof the sensor, and a projector configured to execute projecting thefirst image on the projection surface, and projecting a second imageobtained by moving a position of the at least one first control pointfrom a first position to a second position based on a position of atleast one of the hands of the user in the first image on the projectionsurface when the second gesture operation is detected by the controldevice.

According to the above, there are exerted substantially the sameadvantages as those of the method of correcting the projection imageaccording to Supplementary Note 1.

Supplementary Note 9

A non-transitory computer-readable storage medium storing a programconfigured to make at least one processor execute processing includingprojecting a first image including a plurality of control pointsconfigured to correct a projection image from a projector on aprojection surface, identifying at least one first control point fromthe plurality of control points based on a position of at least one ofhands of a user in the first image when a first gesture operation by atleast one of the hands of the user is detected based on an output valueof a sensor, and projecting a second image obtained by moving a positionof the at least one first control point from a first position to asecond position based on a position of at least one of the hands of theuser in the first image from the projector on the projection surfacewhen a second gesture operation by at least one of the hands of the userdifferent from the first gesture operation is detected based on anoutput value of the sensor.

According to the above, there are exerted substantially the sameadvantages as those of the method of correcting the projection imageaccording to Supplementary Note 1.

What is claimed is:
 1. A method of correcting a projection image, themethod comprising: projecting, by a projector, a first image including aplurality of control points configured to correct the projection imageon a projection surface; identifying at least one first control pointfrom the plurality of control points based on a position of at least oneof hands of a user in the first image when a first gesture operation byat least one of the hands of the user is detected based on an outputvalue of a sensor; and projecting, by the projector, a second imageobtained by moving a position of the at least one first control pointfrom a first position to a second position based on a position of atleast one of the hands of the user in the first image on the projectionsurface when a second gesture operation by at least one of the hands ofthe user different from the first gesture operation is detected based onan output value of the sensor.
 2. The method of correcting theprojection image according to claim 1, wherein the first gestureoperation is an operation of moving at least one of the hands of theuser having a first shape into a range in which a distance from theprojection surface is equal to or shorter than a threshold value, andthe second gesture operation is an operation of moving at least one ofthe hands of the user having a second shape different from the firstshape in the range as much as a distance equal to or longer than a firstdistance.
 3. The method of correcting the projection image according toclaim 2, wherein the identifying at least one first control pointincludes when a third gesture operation is detected based on an outputvalue of the sensor, identifying a setting area set by the third gestureoperation in the first image, and identifying a control point includedin the setting area out of the plurality of control points as the atleast one first control point, and the third gesture operation is anoperation of moving at least one of the hands of the user having thefirst shape in the range as much as a distance equal to or longer than asecond distance.
 4. The method of correcting the projection imageaccording to claim 2, further comprising: changing the at least onefirst control point to a second control point when a fourth gestureoperation is detected based on an output value of the sensor, whereinthe fourth gesture operation is an operation of moving at least one ofthe hands of the user having the first shape into the range, and thenmoving the at least one of the hands of the user having the first shapeoutside the range within a predetermined period.
 5. The method ofcorrecting the projection image according to claim 4, wherein the firstcontrol point is a control point in a first display mode, and the secondcontrol point is a control point in a second display mode different fromthe first display mode.
 6. The method of correcting the projection imageaccording to claim 2, wherein the first shape is a state of erecting Xfingers, the second shape is a state of erecting Y fingers, the X is aninteger that is equal to or more than 0 and that is equal to or lessthan 5, the Y is an integer that is equal to or more than 0 and that isequal to or less than 5, and the X is different from the Y.
 7. Aprojection system comprising: a sensor; a control device configured toexecute identifying at least one first control point from a plurality ofcontrol points configured to correct the projection image based on aposition of at least one of hands of a user in a first image which isprojected on a projection surface, and which includes the plurality ofcontrol points, when a first gesture operation by at least one of thehands of the user is detected based on an output value of the sensor,and detecting a second gesture operation by at least one of the hands ofthe user different from the first gesture operation based on an outputvalue of the sensor; and a projector configured to execute projectingthe first image on the projection surface, and projecting a second imageobtained by moving a position of the at least one first control pointfrom a first position to a second position based on a position of atleast one of the hands of the user in the first image on the projectionsurface when the second gesture operation is detected by the controldevice.
 8. A non-transitory computer-readable storage medium storing aprogram, the program being configured to make at least one processorexecute processing comprising: projecting a first image including aplurality of control points configured to correct a projection imagefrom a projector on a projection surface; identifying at least one firstcontrol point from the plurality of control points based on a positionof at least one of hands of a user in the first image when a firstgesture operation by at least one of the hands of the user is detectedbased on an output value of a sensor; and projecting a second imageobtained by moving a position of the at least one first control pointfrom a first position to a second position based on a position of atleast one of the hands of the user in the first image from the projectoron the projection surface when a second gesture operation by at leastone of the hands of the user different from the first gesture operationis detected based on an output value of the sensor.